1. Field of the Invention
The present invention relates generally to the field of facial expression recognition, and more particularly, to a method and an apparatus for recognizing an emotion of an individual using facial Action Units (AUs).
2. Description of the Related Art
In life, people often express emotions through facial expressions. Facial expressions are some of the most powerful, natural, and immediate ways for humans to communicate their emotions and intentions. The face can express an emotion sooner than people verbalize or even realize their feelings. For example, different emotions are expressed using various facial regions, mainly the mouth, the eyes, and the eyebrows.
More often, emotional expression is communicated by subtle changes in one or a few discrete facial features, such as a tightening of the lips in anger or obliquely opening the lip corners in sadness. Many computer systems are configured to recognize a small set of prototypic emotional expressions, e.g., joy, surprise, anger, sadness, fear, and disgust.
A Facial Action Coding System (FACS) has been developed for describing facial expressions by AUs. Of the 44 FACS AUs defined in the FACS, 30 AUs are anatomically related to contractions of specific facial muscles, i.e., 12 are for upper face, and 18 are for lower face. AUs can occur either singly or in combination. When AUs occur in combination, they may be additive, in which the combination does not change the appearance of the constituent Aus, or non-additive, in which the appearance of the constituents does change.
As an example of a non-additive effect, AU 4 appears differently depending on whether it occurs alone or in combination with AU 1 (as in AU 1+4). When AU 4 occurs alone, the brows are drawn together and lowered. In AU 1+4, the brows are drawn together but are raised due to the action of AU 1. AU 1+2 is another example of non-additive combinations. When AU 2 occurs alone, it not only raises the outer brow, but also often pulls up the inner brow, which results in a very similar appearance to AU 1+2. These effects of the non-additive AU combinations increase the difficulties of AU recognition.
Current systems generally adopt a single phase method or two phase method for recognizing facial expressions using AUs. The two phase method is more practical in analyzing emotions than the single phase method. In the two phase method, a set of action units describing a facial expression of an individual in an image or a video frame is detected. The set of action units are then mapped to one or more target emotions to recognize an actual emotion of the individual. Typically, the two phase method uses formulated mapping rules for mapping a set of action units to target emotions. The formulated mapping rules are based on an Emotional Facial Action Coding System and are sensitive to noise in the input. However, the two phase method fails to compensate for weaknesses in the current AU detection techniques.
Currently known AU detection methods suffer from errors due to certainty in accurately tracking a facial expression and extracting features from the facial expression. When such erroneous action units are fed for mapping with target emotions using the two phase method, the accuracy of the final emotion recognized based on the erroneous action units often deteriorates.